{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Question_vs_Statement.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tOHVDa9DQQR5"
      },
      "source": [
        "![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/component_examples/classifiers/Question_vs_Statement.ipynb)\n",
        "\n",
        "\n",
        "# [Question vs Statement Classifier](https://nlp.johnsnowlabs.com/2021/11/04/bert_sequence_classifier_question_statement_en.html)\n",
        "\n",
        "\n",
        "Trained to add sentence classifying capabilities to distinguish between Question vs Statements.\n",
        "\n",
        "This model was imported from [Hugging Face](https://huggingface.co/shahrukhx01/question-vs-statement-classifier), and trained based on [Haystack](https://github.com/deepset-ai/haystack/issues/611).\n",
        "\n",
        "\n",
        "This model can be used to classify questions in comments and automatically add the appropirate label\n",
        "<br>\n",
        "\n",
        "This model can be used to predict the following news categories \n",
        "`question`, `statement`\n",
        "\n",
        "<br>\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "The data Source used to train this can be found [here](https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs)\n",
        "\n",
        "<br>\n",
        "\n",
        "##Benchmark on Dataset \n",
        "![image.png]()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##1.Setup Java 8 and NLU"
      ],
      "metadata": {
        "id": "HIxATMI7ixJx"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SF5-Z-U4jukd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fb8151e4-ecdd-4a71-8075-2a3a1fb626c0"
      },
      "source": [
        "!wget https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh -O - | bash\n",
        "import nlu"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-05-19 23:27:38--  https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1665 (1.6K) [text/plain]\n",
            "Saving to: ‘STDOUT’\n",
            "\n",
            "-                     0%[                    ]       0  --.-KB/s               Installing  NLU 3.4.4rc1 with  PySpark 3.0.3 and Spark NLP 3.4.3 for Google Colab ...\n",
            "-                   100%[===================>]   1.63K  --.-KB/s    in 0.001s  \n",
            "\n",
            "2022-05-19 23:27:38 (1.43 MB/s) - written to stdout [1665/1665]\n",
            "\n",
            "Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease\n",
            "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]\n",
            "Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]\n",
            "Hit:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease\n",
            "Ign:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease\n",
            "Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]\n",
            "Hit:7 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release\n",
            "Hit:8 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease\n",
            "Hit:9 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease\n",
            "Get:10 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease [3,626 B]\n",
            "Hit:11 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease\n",
            "Get:12 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease [21.3 kB]\n",
            "Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [966 kB]\n",
            "Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,277 kB]\n",
            "Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,199 kB]\n",
            "Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [29.8 kB]\n",
            "Get:18 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [22.8 kB]\n",
            "Get:19 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,765 kB]\n",
            "Get:20 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,503 kB]\n",
            "Get:21 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [932 kB]\n",
            "Get:22 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic/main amd64 Packages [44.3 kB]\n",
            "Fetched 12.0 MB in 4s (3,305 kB/s)\n",
            "Reading package lists... Done\n",
            "tar: spark-3.0.2-bin-hadoop2.7.tgz: Cannot open: No such file or directory\n",
            "tar: Error is not recoverable: exiting now\n",
            "\u001b[K     |████████████████████████████████| 209.1 MB 56 kB/s \n",
            "\u001b[K     |████████████████████████████████| 144 kB 21.7 MB/s \n",
            "\u001b[K     |████████████████████████████████| 517 kB 44.6 MB/s \n",
            "\u001b[K     |████████████████████████████████| 198 kB 56.3 MB/s \n",
            "\u001b[?25h  Building wheel for pyspark (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting nlu_tmp==3.4.4rc9\n",
            "  Downloading nlu_tmp-3.4.4rc9-py3-none-any.whl (528 kB)\n",
            "\u001b[K     |████████████████████████████████| 528 kB 4.9 MB/s \n",
            "\u001b[?25hCollecting spark-nlp<3.5.0,>=3.4.4\n",
            "  Downloading spark_nlp-3.4.4-py2.py3-none-any.whl (145 kB)\n",
            "\u001b[K     |████████████████████████████████| 145 kB 53.0 MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (1.21.6)\n",
            "Requirement already satisfied: pandas>=1.3.5 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (1.3.5)\n",
            "Requirement already satisfied: pyarrow>=0.16.0 in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (6.0.1)\n",
            "Requirement already satisfied: dataclasses in /usr/local/lib/python3.7/dist-packages (from nlu_tmp==3.4.4rc9) (0.6)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.3.5->nlu_tmp==3.4.4rc9) (2022.1)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.3.5->nlu_tmp==3.4.4rc9) (2.8.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=1.3.5->nlu_tmp==3.4.4rc9) (1.15.0)\n",
            "Installing collected packages: spark-nlp, nlu-tmp\n",
            "  Attempting uninstall: spark-nlp\n",
            "    Found existing installation: spark-nlp 3.4.3\n",
            "    Uninstalling spark-nlp-3.4.3:\n",
            "      Successfully uninstalled spark-nlp-3.4.3\n",
            "Successfully installed nlu-tmp-3.4.4rc9 spark-nlp-3.4.4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##2.Load the mdoel and make Sample Predictions "
      ],
      "metadata": {
        "id": "XiVdjjfzij2R"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pipeline = nlu.load('en.classify.question_vs_statement')\n",
        "pipeline.predict(\"What feature in your car did you not realize you had until someone else told you about it?\",output_level = 'document')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 133
        },
        "id": "zzWPoMYZsVj2",
        "outputId": "91aeab78-aa6c-40f6-e078-0da7c284a079"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "bert_sequence_classifier_question_statement download started this may take some time.\n",
            "Approximate size to download 40.4 MB\n",
            "[OK!]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  classified_sequence classified_sequence_confidence  \\\n",
              "0            question                         0.9773   \n",
              "\n",
              "                                            document  \n",
              "0  What feature in your car did you not realize y...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b94b623d-6f37-4853-8ab1-8e34533b7c23\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>classified_sequence</th>\n",
              "      <th>classified_sequence_confidence</th>\n",
              "      <th>document</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>question</td>\n",
              "      <td>0.9773</td>\n",
              "      <td>What feature in your car did you not realize y...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b94b623d-6f37-4853-8ab1-8e34533b7c23')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b94b623d-6f37-4853-8ab1-8e34533b7c23 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b94b623d-6f37-4853-8ab1-8e34533b7c23');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##3.Define Sample Sentences"
      ],
      "metadata": {
        "id": "_XcMyXVXsdYW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sample_sentences = [\n",
        "\"I love the food here!\",\n",
        "\"Which movie genre would you like to watch?\",\n",
        "\"Wanna go out tonight?\",\n",
        "\"I am going to get the new game.\"\n",
        "]"
      ],
      "metadata": {
        "id": "c_ty0gP6sglE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "##4.Predict on Sample Sentences"
      ],
      "metadata": {
        "id": "qpD0_HqksiLw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pipeline.predict(sample_sentences,output_level = 'document')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "zynQvfNmWI50",
        "outputId": "2feae62d-d8e8-42a6-d92b-349a073fd9c5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  classified_sequence classified_sequence_confidence  \\\n",
              "0           statement                       0.996214   \n",
              "1            question                       0.998511   \n",
              "2            question                       0.993408   \n",
              "3           statement                       0.998788   \n",
              "\n",
              "                                     document  \n",
              "0                       I love the food here!  \n",
              "1  Which movie genre would you like to watch?  \n",
              "2                       Wanna go out tonight?  \n",
              "3             I am going to get the new game.  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d6b5521d-8a14-4962-81ce-3a435e3fdf05\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>classified_sequence</th>\n",
              "      <th>classified_sequence_confidence</th>\n",
              "      <th>document</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>statement</td>\n",
              "      <td>0.996214</td>\n",
              "      <td>I love the food here!</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>question</td>\n",
              "      <td>0.998511</td>\n",
              "      <td>Which movie genre would you like to watch?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>question</td>\n",
              "      <td>0.993408</td>\n",
              "      <td>Wanna go out tonight?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>statement</td>\n",
              "      <td>0.998788</td>\n",
              "      <td>I am going to get the new game.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d6b5521d-8a14-4962-81ce-3a435e3fdf05')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-d6b5521d-8a14-4962-81ce-3a435e3fdf05 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-d6b5521d-8a14-4962-81ce-3a435e3fdf05');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##5.Take a look at the parmaters of the pipeline"
      ],
      "metadata": {
        "id": "cEf6CWsxtDWJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pipeline.print_info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3TgLjxNltPCp",
        "outputId": "8b871257-c613-4c1d-de2a-46bd8d2bc030"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The following parameters are configurable for this NLU pipeline (You can copy paste the examples) :\n",
            ">>> component_list['bert_for_sequence_classification'] has settable params:\n",
            "component_list['bert_for_sequence_classification'].setActivation('softmax')  | Info: Whether to calculate logits via Softmax or Sigmoid. Default is Softmax | Currently set to : softmax\n",
            "component_list['bert_for_sequence_classification'].setBatchSize(8)  | Info: Size of every batch | Currently set to : 8\n",
            "component_list['bert_for_sequence_classification'].setCoalesceSentences(False)  | Info: Instead of 1 class per sentence (if inputCols is '''sentence''') output 1 class per document by averaging probabilities in all sentences. | Currently set to : False\n",
            "component_list['bert_for_sequence_classification'].setMaxSentenceLength(128)  | Info: Max sentence length to process | Currently set to : 128\n",
            "component_list['bert_for_sequence_classification'].setCaseSensitive(True)  | Info: whether to ignore case in tokens for embeddings matching | Currently set to : True\n",
            ">>> component_list['tokenizer'] has settable params:\n",
            "component_list['tokenizer'].setTargetPattern('\\S+')            | Info: pattern to grab from text as token candidates. Defaults \\S+ | Currently set to : \\S+\n",
            "component_list['tokenizer'].setContextChars(['.', ',', ';', ':', '!', '?', '*', '-', '(', ')', '\"', \"'\"])  | Info: character list used to separate from token boundaries | Currently set to : ['.', ',', ';', ':', '!', '?', '*', '-', '(', ')', '\"', \"'\"]\n",
            "component_list['tokenizer'].setCaseSensitiveExceptions(True)   | Info: Whether to care for case sensitiveness in exceptions | Currently set to : True\n",
            "component_list['tokenizer'].setMinLength(0)                    | Info: Set the minimum allowed legth for each token | Currently set to : 0\n",
            "component_list['tokenizer'].setMaxLength(99999)                | Info: Set the maximum allowed legth for each token | Currently set to : 99999\n",
            ">>> component_list['document_assembler'] has settable params:\n",
            "component_list['document_assembler'].setCleanupMode('shrink')  | Info: possible values: disabled, inplace, inplace_full, shrink, shrink_full, each, each_full, delete_full | Currently set to : shrink\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Looking Good! Let's test this model on a labelled dataset to see how it performs "
      ],
      "metadata": {
        "id": "1hF8dmLIkbSz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##6.Download Data\n",
        "\n",
        "we are going to test the model on [this](https://www.kaggle.com/datasets/stefanondisponibile/quora-question-keyword-pairs?select=test.tsv) dataset \n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "uRbusvKOkVac"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget http://ckl-it.de/wp-content/uploads/2022/05/question_vs_statement.csv"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KvD0rZTBrnrL",
        "outputId": "4a4097c6-8e56-4efe-83d6-f98909cfb50a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-05-19 23:43:13--  http://ckl-it.de/wp-content/uploads/2022/05/question_vs_statement.csv\n",
            "Resolving ckl-it.de (ckl-it.de)... 217.160.0.108, 2001:8d8:100f:f000::209\n",
            "Connecting to ckl-it.de (ckl-it.de)|217.160.0.108|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 9931364 (9.5M) [text/csv]\n",
            "Saving to: ‘question_vs_statement.csv’\n",
            "\n",
            "question_vs_stateme 100%[===================>]   9.47M  9.06MB/s    in 1.0s    \n",
            "\n",
            "2022-05-19 23:43:14 (9.06 MB/s) - ‘question_vs_statement.csv’ saved [9931364/9931364]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas  as pd \n",
        "df = pd.read_csv(\"question_vs_statement.csv\")\n",
        "df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "n-6bOT4krpr8",
        "outputId": "f57b1d19-3acb-4b19-ac8f-66021a7fd3ba"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        Unnamed: 0                                               text  \\\n",
              "0                0            Why are African-Americans so beautiful?   \n",
              "1                1                                  African Americans   \n",
              "2                2                    Why are hispanics so beautiful?   \n",
              "3                3                                hispanics beautiful   \n",
              "4                4  I want to pursue PhD in Computer Science about...   \n",
              "...            ...                                                ...   \n",
              "160988      160988                          calculate tension physics   \n",
              "160989      160989     How can I make money online quickly and easily   \n",
              "160990      160990                                              money   \n",
              "160991      160991                         What is make money online?   \n",
              "160992      160992                                    What make money   \n",
              "\n",
              "                y  \n",
              "0        question  \n",
              "1       statement  \n",
              "2        question  \n",
              "3       statement  \n",
              "4        question  \n",
              "...           ...  \n",
              "160988  statement  \n",
              "160989   question  \n",
              "160990  statement  \n",
              "160991   question  \n",
              "160992  statement  \n",
              "\n",
              "[160993 rows x 3 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8e4389b1-343a-4ce3-b573-13ffbb81ad66\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>text</th>\n",
              "      <th>y</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>Why are African-Americans so beautiful?</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>African Americans</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>Why are hispanics so beautiful?</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>hispanics beautiful</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>I want to pursue PhD in Computer Science about...</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>160988</th>\n",
              "      <td>160988</td>\n",
              "      <td>calculate tension physics</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>160989</th>\n",
              "      <td>160989</td>\n",
              "      <td>How can I make money online quickly and easily</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>160990</th>\n",
              "      <td>160990</td>\n",
              "      <td>money</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>160991</th>\n",
              "      <td>160991</td>\n",
              "      <td>What is make money online?</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>160992</th>\n",
              "      <td>160992</td>\n",
              "      <td>What make money</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>160993 rows × 3 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8e4389b1-343a-4ce3-b573-13ffbb81ad66')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8e4389b1-343a-4ce3-b573-13ffbb81ad66 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8e4389b1-343a-4ce3-b573-13ffbb81ad66');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's take  a Peek at the distribution of the labels "
      ],
      "metadata": {
        "id": "mUl3_RmhuSHW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df.y.value_counts().plot.barh(title='Distribution of Labels')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "n_c6k0EaNpM9",
        "outputId": "ddc9a163-2244-43ac-aef4-fbe43999c830"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f377d85dc10>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEICAYAAACTVrmbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUJElEQVR4nO3df7RdZX3n8fdHguFHMEhDmYBKoE1V8AdG/EFV1NaiEKuuDqvg2BbULip27Fi1Fgo60LEjFeuitk4RHUv9hSBTWwurBXWwIloxgQBhBESI/AiCaImAlMWP7/yxn+DJ4Sa5SZ7knoT3a62z7j7P3vvZ33POvudz9vOcm6SqkCSpp8fNdAGSpG2P4SJJ6s5wkSR1Z7hIkrozXCRJ3RkukqTuDBdtM5KcnuQ9nfp6SpJ7kmzX7n81ye/26Lv1989JjurV3wYc931J7kzyg459LkhSSWZtyX012QwXbRWSrEhyX5K7k9yV5BtJ3pLkkXO4qt5SVf9jmn29Yl3bVNVNVTWnqh7qUPtJST491v+hVfV3m9r3BtbxFOCdwH5V9Z+mWP+yJLdsyZq07TJctDX59araBdgbOAX4Y+B/9z7INvwp+inAj6rqjpkuRNs+w0VbnapaVVVfBI4AjkryDIAkZyZ5X1uel+S8dpXz4yQXJ3lckk8xvMn+Uxv2evfI0Mybk9wE/N+1DNf8QpJLk/wkyT8m2a0d61Gf+FdfHSV5FfAnwBHteFe09Y8Ms7W6Tkzy/SR3JPlkkrlt3eo6jkpyUxvSOmFtz02SuW3/H7b+Tmz9vwL4ErBnq+PMDXnOkyxOcnl77DcnOWmKzd6UZGWS25K8a2TfxyU5Lsn3kvwoyTmrn7spjnN0khvaFeqNSd6wIXVqchgu2mpV1aXALcBLplj9zrZud2APhjf4qqrfBm5iuAqaU1UfGNnnpcDTgVeu5ZC/A7wJmA88CHx4GjX+C/A/gbPb8Z49xWZHt9vLgX2BOcBfj23zYuCpwK8C703y9LUc8q+Aua2fl7aa31hVXwYOBVa2Oo5eX+1j7m197QosBo5N8rqxbV4OLAQOAf54ZOjxbcDrWj17Av8OfGT8AEl2ZnhOD21XqL8MLNvAOjUhDBdt7VYCU30KfoAhBPauqgeq6uJa/z+kd1JV3VtV961l/aeqanlV3Qu8B/jN1RP+m+gNwIeq6oaqugc4Hjhy7Krp5Kq6r6quAK4AHhVSrZYjgeOr6u6qWgH8BfDbm1pgVX21qq6qqoer6krgLIawGHVye/6uAv4WeH1rfwtwQlXdUlX3AycBh69l+PFh4BlJdqyq26rq6k2tXTPDcNHWbi/gx1O0nwpcD1zYhlmOm0ZfN2/A+u8D2wPzplXluu3Z+hvtexbDFddqo9/u+inD1c24ea2m8b722tQCk7wgyUVtuG0VQ2CMP/bx52fPtrw38IU2RHkX8B3gIdZ8fLTQPqL1fVuS85M8bVNr18wwXLTVSvI8hjfOr4+va5/c31lV+wKvAd6R5FdXr15Ll+u7snnyyPJTGK6O7mQYMtpppK7tGIbjptvvSoY34NG+HwRuX89+4+5sNY33desG9jOVzwJfBJ5cVXOB04GMbTP+/KxsyzczDHXtOnLboaoeVVdVXVBVv8Zw1XkN8LEOtWsGGC7a6iR5QpJXA58DPt2GYca3eXWSX0wSYBXDJ+WH2+rbGeYkNtRvJdkvyU7AnwLntq8qXwfs0Ca9twdOBGaP7Hc7sGD0a9NjzgL+MMk+SebwszmaBzekuFbLOcCfJdklyd7AO4BPr3vPNSXZYewWYBfgx1X1H0meD/yXKXZ9T5KdkuwPvBE4u7Wf3mrau/W/e5LXTnHcPZK8ts293A/cw89eM21lDBdtTf4pyd0Mn4RPAD7E8CY2lYXAlxneoL4J/K+quqitez9wYhumedda9p/Kp4AzGYaodgD+AIZvrwFvBT7OcJVwL8OXCVb7fPv5oySXTdHvJ1rfXwNuBP6DYRJ8Y7ytHf8Ghiu6z7b+p2sv4L6x2y8wPL4/bc//exlCbNy/MgxFfgX4YFVd2Nr/kuGq58K2/78BL5hi/8cxhOFKhqHOlwLHbkDtmiDxPwuTJPXmlYskqTvDRZLUneEiSerOcJEkdbet/gN9G2TevHm1YMGCmS5DkrYqS5cuvbOqdp9qneECLFiwgCVLlsx0GZK0VUny/bWtc1hMktSd4SJJ6s5wkSR1Z7hIkrozXCRJ3RkukqTuDBdJUneGiySpO8NFktSd4SJJ6s5wkSR1Z7hIkrozXCRJ3RkukqTuDBdJUneGiySpO8NFktSd4SJJ6s5wkSR1Z7hIkrozXCRJ3RkukqTuDBdJUneGiySpO8NFktSd4SJJ6m7WTBcwCa66dRULjjt/psuQpM1uxSmLt8hxvHKRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKk7w0WS1J3hIknqznCRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKk7w0WS1J3hIknqznCRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKk7w0WS1N1GhUuStyfZqdd2m0uSo5PsOVPHl6THqo29cnk7MJ3QmO52m8vRgOEiSVvYesMlyc5Jzk9yRZLlSf47wxv2RUkuatv8TZIlSa5OcnJr+4MptjskyTeTXJbk80nmtPYVSd6fZFnrZ1GSC5J8L8lbRmr5oyTfTnLlyHEWJPlOko+141+YZMckhwMHAp9p/e7Y96mTJK3NdK5cXgWsrKpnV9UzgNOAlcDLq+rlbZsTqupA4FnAS5M8q6o+PLpdknnAicArqmoRsAR4x8hxbqqqA4CLgTOBw4EXAqtD5BBgIfB84ADguUkObvsuBD5SVfsDdwH/uarObcd4Q1UdUFX3jT6oJMe0IFvy0E9XTfPpkiRNx6xpbHMV8BdJ/hw4r6ouTjK+zW8mOab1Nx/YD7hybJsXtvZL2v6PB745sv6LI8ebU1V3A3cnuT/JrsAh7XZ5224OQ6jcBNxYVcta+1JgwfoeVFWdAZwBMHv+wlrf9pKk6VtvuFTVdUkWAYcB70vyldH1SfYB3gU8r6r+PcmZwA5TdBXgS1X1+rUc6v728+GR5dX3Z7X9319VHx07/oKx7R8CHAKTpBk0nTmXPYGfVtWngVOBRcDdwC5tkycA9wKrkuwBHDqy++h2/wa8KMkvtn53TvJLG1DrBcCbRuZp9kry8+vZZ/T4kqQtZDrDYs8ETk3yMPAAcCxwEPAvSVa2+ZTLgWuAm4FLRvY9Y2y7o4Gzksxu608ErptOoVV1YZKnA99sw2r3AL/FcKWyNmcCpye5DzhofN5FkrR5pMrphtnzF9b8o06b6TIkabNbccribn0lWdq+zPUo/oW+JKk7w0WS1J3hIknqznCRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKk7w0WS1J3hIknqznCRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKk7w0WS1J3hIknqznCRJHVnuEiSups10wVMgmfuNZclpyye6TIkaZvhlYskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6mzXTBUyCq25dxYLjzp/pMiRpi1pxyuLN1rdXLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6m/hwSfInY/e/MVO1SJKmZ+LDBVgjXKrql2eqEEnS9HQJlyQnJLkuydeTnJXkXUm+muTAtn5ekhVtebskpyb5dpIrk/xea5+f5GtJliVZnuQlSU4Bdmxtn2nb3dN+pvWzPMlVSY5o7S9rxz43yTVJPpMkPR6nJGl6Zm1qB0meCxwJHND6uwxYuo5d3gysqqrnJZkNXJLkQuA3gAuq6s+SbAfsVFUXJ/mvVXXAFP38Rjvms4F5wLeTfK2tew6wP7ASuAR4EfD1sbqPAY4B2O4Ju2/EI5ckrc0mhwvwEuALVfVTgCRfXM/2hwDPSnJ4uz8XWAh8G/hEku2Bf6iqZevp58XAWVX1EHB7kn8Fngf8BLi0qm5p9SwDFjAWLlV1BnAGwOz5C2s6D1SSND09wmVtHuRnw247jLQHeFtVXTC+Q5KDgcXAmUk+VFWf3Mhj3z+y/BCb93FKksb0mHP5GvC6JDsm2QX49da+AnhuWz58ZPsLgGPbFQpJfinJzkn2Bm6vqo8BHwcWte0fWL3tmIuBI9oczu7AwcClHR6PJGkTbfIn+qq6LMnZwBXAHQzDWwAfBM5pcxvnj+zycYZhqsvaRPsPgdcBLwP+KMkDwD3A77TtzwCuTHJZVb1hpJ8vAAe14xbw7qr6QZKnbepjkiRtmlT1nW5IchJwT1V9sGvHm9Hs+Qtr/lGnzXQZkrRFrThl8Sbtn2RpVR041bqt4e9cJElbme4T3VV1Uu8+JUlbF69cJEndGS6SpO4MF0lSd4aLJKk7w0WS1J3hIknqznCRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKk7w0WS1J3hIknqznCRJHVnuEiSujNcJEndGS6SpO4MF0lSd4aLJKm7WTNdwCR45l5zWXLK4pkuQ5K2GV65SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd0ZLpKk7gwXSVJ3hoskqTvDRZLUneEiSerOcJEkdWe4SJK6M1wkSd2lqma6hhmX5G7g2pmuYx3mAXfOdBFrMcm1wWTXN8m1wWTXZ20br2d9e1fV7lOtmNXpAFu7a6vqwJkuYm2SLJnU+ia5Npjs+ia5Npjs+qxt422p+hwWkyR1Z7hIkrozXAZnzHQB6zHJ9U1ybTDZ9U1ybTDZ9Vnbxtsi9TmhL0nqzisXSVJ3hoskqbvHfLgkeVWSa5Ncn+S4zXicTyS5I8nykbbdknwpyXfbzye29iT5cKvpyiSLRvY5qm3/3SRHjbQ/N8lVbZ8PJ8kG1PbkJBcl+X9Jrk7y3yasvh2SXJrkilbfya19nyTfan2eneTxrX12u399W79gpK/jW/u1SV450r5J50GS7ZJcnuS8CaxtRXvulyVZ0tom5bXdNcm5Sa5J8p0kB01QbU9tz9nq20+SvH2C6vvD9vuwPMlZGX5PJua8o6oeszdgO+B7wL7A44ErgP0207EOBhYBy0faPgAc15aPA/68LR8G/DMQ4IXAt1r7bsAN7ecT2/IT27pL27Zp+x66AbXNBxa15V2A64D9Jqi+AHPa8vbAt1pf5wBHtvbTgWPb8luB09vykcDZbXm/9hrPBvZpr/12Pc4D4B3AZ4Hz2v1Jqm0FMG+sbVJe278DfrctPx7YdVJqm+K94gfA3pNQH7AXcCOw48j5dvREnXcb80RvKzfgIOCCkfvHA8dvxuMtYM1wuRaY35bnM/wxJ8BHgdePbwe8HvjoSPtHW9t84JqR9jW224g6/xH4tUmsD9gJuAx4AcNfGc8afy2BC4CD2vKstl3GX9/V223qeQA8CfgK8CvAee1YE1Fb22cFjw6XGX9tgbkMb5CZtNqmqPUQ4JJJqY8hXG5mCKxZ7bx75SSdd4/1YbHVL9Bqt7S2LWWPqrqtLf8A2GM9da2r/ZYp2jdYu1x+DsPVwcTUl2HYaRlwB/Alhk9Vd1XVg1P0+Ugdbf0q4Oc2ou7pOg14N/Bwu/9zE1QbQAEXJlma5JjWNgmv7T7AD4G/zTCk+PEkO09IbeOOBM5qyzNeX1XdCnwQuAm4jeE8WsoEnXeP9XCZGDV8PJjR74UnmQP8H+DtVfWT0XUzXV9VPVRVBzBcJTwfeNpM1TIqyauBO6pq6UzXsg4vrqpFwKHA7yc5eHTlDL62sxiGiv+mqp4D3MswzDQJtT2izVu8Bvj8+LqZqq/N87yWIaD3BHYGXrWl61iXx3q43Ao8eeT+k1rblnJ7kvkA7ecd66lrXe1PmqJ92pJszxAsn6mqv5+0+larqruAixgu23dNsvrfxxvt85E62vq5wI82ou7peBHwmiQrgM8xDI395YTUBjzyKZequgP4AkM4T8JrewtwS1V9q90/lyFsJqG2UYcCl1XV7e3+JNT3CuDGqvphVT0A/D3DuTgx591GjclvKzeGT043MKT/6kmr/Tfj8Raw5pzLqaw5MfiBtryYNScGL23tuzGMUT+x3W4EdmvrxicGD9uAugJ8EjhtrH1S6tsd2LUt7whcDLya4ZPk6OTlW9vy77Pm5OU5bXl/1py8vIFh4rLLeQC8jJ9N6E9EbQyfaHcZWf4GwyfcSXltLwae2pZPanVNRG0jNX4OeOMk/V4wzDlezTAHGYYvRrxtUs67qsf4hH57cg9j+HbU94ATNuNxzmIYG32A4RPbmxnGPL8CfBf48sgJF+AjraargANH+nkTcH27jZ7wBwLL2z5/zdgk6XpqezHDpf2VwLJ2O2yC6nsWcHmrbznw3ta+b/vlvL79Us1u7Tu0+9e39fuO9HVCq+FaRr6Z0+M8YM1wmYjaWh1XtNvVq/efoNf2AGBJe23/geHNdyJqa/vvzPAJf+5I20TUB5wMXNP2/xRDQEzEeVdV/vMvkqT+HutzLpKkzcBwkSR1Z7hIkrozXCRJ3RkukqTuDBdJUneGiySpu/8PK4DF9wYhhk4AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##7.Make Predictions with the model"
      ],
      "metadata": {
        "id": "KHxOy5o9uyTG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# The dataset is very large so we only take the first 15000 rows\n",
        "df = df.iloc[:15000]"
      ],
      "metadata": {
        "id": "SnWo1ID5Pji2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "predctions = pipeline.predict(df,output_level = 'document')"
      ],
      "metadata": {
        "id": "QpmPv2q5uVyd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "predctions = predctions.dropna(subset = ['classified_sequence'])"
      ],
      "metadata": {
        "id": "rvyeXaHfL9PP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "predctions"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 815
        },
        "id": "j4JpnxjzMOsV",
        "outputId": "ae0494b7-3e9d-4b62-e38d-4a93431cce2c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       Unnamed: 0 classified_sequence classified_sequence_confidence  \\\n",
              "0               0            question                        0.99905   \n",
              "1               1           statement                       0.998174   \n",
              "2               2            question                       0.999225   \n",
              "3               3           statement                       0.995106   \n",
              "4               4            question                       0.998553   \n",
              "...           ...                 ...                            ...   \n",
              "14995       14995           statement                       0.995937   \n",
              "14996       14996            question                       0.999357   \n",
              "14997       14997            question                       0.999604   \n",
              "14998       14998            question                       0.999142   \n",
              "14999       14999            question                        0.63007   \n",
              "\n",
              "                                                document  \\\n",
              "0                Why are African-Americans so beautiful?   \n",
              "1                                      African Americans   \n",
              "2                        Why are hispanics so beautiful?   \n",
              "3                                    hispanics beautiful   \n",
              "4      I want to pursue PhD in Computer Science about...   \n",
              "...                                                  ...   \n",
              "14995                        review NumPy Beginner Guide   \n",
              "14996  What are some reasons why marijuana should be ...   \n",
              "14997                     why marijuana medical purposes   \n",
              "14998  What is the reason why marijuana should be leg...   \n",
              "14999                  reason marijuana medical purposes   \n",
              "\n",
              "                                                    text          y  \n",
              "0                Why are African-Americans so beautiful?   question  \n",
              "1                                      African Americans  statement  \n",
              "2                        Why are hispanics so beautiful?   question  \n",
              "3                                    hispanics beautiful  statement  \n",
              "4      I want to pursue PhD in Computer Science about...   question  \n",
              "...                                                  ...        ...  \n",
              "14995                        review NumPy Beginner Guide  statement  \n",
              "14996  What are some reasons why marijuana should be ...   question  \n",
              "14997                     why marijuana medical purposes  statement  \n",
              "14998  What is the reason why marijuana should be leg...   question  \n",
              "14999                  reason marijuana medical purposes  statement  \n",
              "\n",
              "[15000 rows x 6 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b9ce6ee0-b86f-4d3c-87c3-68a1cbfd3eaa\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>classified_sequence</th>\n",
              "      <th>classified_sequence_confidence</th>\n",
              "      <th>document</th>\n",
              "      <th>text</th>\n",
              "      <th>y</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>question</td>\n",
              "      <td>0.99905</td>\n",
              "      <td>Why are African-Americans so beautiful?</td>\n",
              "      <td>Why are African-Americans so beautiful?</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>statement</td>\n",
              "      <td>0.998174</td>\n",
              "      <td>African Americans</td>\n",
              "      <td>African Americans</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>question</td>\n",
              "      <td>0.999225</td>\n",
              "      <td>Why are hispanics so beautiful?</td>\n",
              "      <td>Why are hispanics so beautiful?</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>statement</td>\n",
              "      <td>0.995106</td>\n",
              "      <td>hispanics beautiful</td>\n",
              "      <td>hispanics beautiful</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>question</td>\n",
              "      <td>0.998553</td>\n",
              "      <td>I want to pursue PhD in Computer Science about...</td>\n",
              "      <td>I want to pursue PhD in Computer Science about...</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14995</th>\n",
              "      <td>14995</td>\n",
              "      <td>statement</td>\n",
              "      <td>0.995937</td>\n",
              "      <td>review NumPy Beginner Guide</td>\n",
              "      <td>review NumPy Beginner Guide</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14996</th>\n",
              "      <td>14996</td>\n",
              "      <td>question</td>\n",
              "      <td>0.999357</td>\n",
              "      <td>What are some reasons why marijuana should be ...</td>\n",
              "      <td>What are some reasons why marijuana should be ...</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14997</th>\n",
              "      <td>14997</td>\n",
              "      <td>question</td>\n",
              "      <td>0.999604</td>\n",
              "      <td>why marijuana medical purposes</td>\n",
              "      <td>why marijuana medical purposes</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14998</th>\n",
              "      <td>14998</td>\n",
              "      <td>question</td>\n",
              "      <td>0.999142</td>\n",
              "      <td>What is the reason why marijuana should be leg...</td>\n",
              "      <td>What is the reason why marijuana should be leg...</td>\n",
              "      <td>question</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14999</th>\n",
              "      <td>14999</td>\n",
              "      <td>question</td>\n",
              "      <td>0.63007</td>\n",
              "      <td>reason marijuana medical purposes</td>\n",
              "      <td>reason marijuana medical purposes</td>\n",
              "      <td>statement</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>15000 rows × 6 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b9ce6ee0-b86f-4d3c-87c3-68a1cbfd3eaa')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b9ce6ee0-b86f-4d3c-87c3-68a1cbfd3eaa button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b9ce6ee0-b86f-4d3c-87c3-68a1cbfd3eaa');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##8.Evaluate Predictions "
      ],
      "metadata": {
        "id": "y03ZPigmGPYL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import classification_report\n",
        "print(classification_report(predctions['y'], predctions['classified_sequence']) )"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yfJtQLGDu8nN",
        "outputId": "9dd782dd-0e3f-4756-b61c-49c530bd0de6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "    question       0.94      0.60      0.73      7536\n",
            "   statement       0.70      0.96      0.81      7464\n",
            "\n",
            "    accuracy                           0.78     15000\n",
            "   macro avg       0.82      0.78      0.77     15000\n",
            "weighted avg       0.82      0.78      0.77     15000\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VXu21c0iQRSC"
      },
      "source": [
        "# There are many more models you can put to use in 1 line of code!\n",
        "## Checkout [the Modelshub](https://nlp.johnsnowlabs.com/models) and the [NLU Namespace](https://nlu.johnsnowlabs.com/docs/en/spellbook) for more models\n",
        "\n",
        "### NLU Webinars and Video Tutorials\n",
        "- [NLU & Streamlit Tutorial](https://vimeo.com/579508034#)\n",
        "- [Crash course of the 50 + Medical Domains and the 200+ Healtchare models in NLU](https://www.youtube.com/watch?v=gGDsZXt1SF8)\n",
        "- [Multi Lingual NLU Webinar - Tutorial on Chinese News dataset](https://www.youtube.com/watch?v=ftAOqJuxnV4)\n",
        "- [John Snow Labs NLU: Become a Data Science Superhero with One Line of Python code](https://events.johnsnowlabs.com/john-snow-labs-nlu-become-a-data-science-superhero-with-one-line-of-python-code?hsCtaTracking=c659363c-2188-4c86-945f-5cfb7b42fcfc%7C8b2b188b-92a3-48ba-ad7e-073b384425b0)\n",
        "- [Python Web Def Conf - Python's NLU library: 1,000+ Models, 200+ Languages, State of the Art Accuracy, 1 Line of Code](https://2021.pythonwebconf.com/presentations/john-snow-labs-nlu-the-simplicity-of-python-the-power-of-spark-nlp)\n",
        "- [NYC/DC NLP Meetup with NLU](https://youtu.be/hJR9m3NYnwk?t=2155)\n",
        "\n",
        "### More ressources \n",
        "- [Join our Slack](https://join.slack.com/t/spark-nlp/shared_invite/zt-lutct9gm-kuUazcyFKhuGY3_0AMkxqA)\n",
        "- [NLU Website](https://nlu.johnsnowlabs.com/)\n",
        "- [NLU Github](https://github.com/JohnSnowLabs/nlu)\n",
        "- [Many more NLU example tutorials](https://github.com/JohnSnowLabs/nlu/tree/master/examples)\n",
        "- [Overview of every powerful nlu 1-liner](https://nlu.johnsnowlabs.com/docs/en/examples)\n",
        "- [Checkout the Modelshub for an overview of all models](https://nlp.johnsnowlabs.com/models) \n",
        "- [Checkout the NLU Namespace where you can find every model as a tabel](https://nlu.johnsnowlabs.com/docs/en/spellbook)\n",
        "- [Intro to NLU article](https://medium.com/spark-nlp/1-line-of-code-350-nlp-models-with-john-snow-labs-nlu-in-python-2f1c55bba619)\n",
        "- [Indepth and easy Sentence Similarity Tutorial, with StackOverflow Questions using BERTology embeddings](https://medium.com/spark-nlp/easy-sentence-similarity-with-bert-sentence-embeddings-using-john-snow-labs-nlu-ea078deb6ebf)\n",
        "- [1 line of Python code for BERT, ALBERT, ELMO, ELECTRA, XLNET, GLOVE, Part of Speech with NLU and t-SNE](https://medium.com/spark-nlp/1-line-of-code-for-bert-albert-elmo-electra-xlnet-glove-part-of-speech-with-nlu-and-t-sne-9ebcd5379cd)"
      ]
    }
  ]
}